Dead Visio process running after completion of publication

Symptoms

In some situations the Visio process is still running after the completion of a publication, or it hangs during a publish and cannot be closed through normal user interaction. This stops further publishes happening, and may also prevent the normal delivery of notification emails.

Visio is a completely independent application of Triaster, although Triaster does of course depend on it. If Visio enters an unresponsive state, the Triaster application can do nothing to recover from the situation.

Cause

Fatal Visio error.

It is not clear what causes Visio to fail in this way. However, the following are known to make the problem more likely:

We are informed by Microsoft that repeated opening of files within Visio (exactly as happens on the Publication Server) is a known source of Visio instability and mitigation and recovery are the best strategy for dealing with it; it seemingly cannot be avoided altogether.
RAM shortage appears to exacerbate the problem.
Failure to apply routine service packs delivered by Microsoft also makes the problem more likely.

Solution

We have observed that the problem is worse on machines where the paging file is used during a publish process, i.e. where the amount of physical RAM in the publication server is less than the amount required. For example, if 3GB are requested during the publish process, but only 2GB are physically available, Visio seems to enter an unstable state more frequently. Consider increasing RAM in the publication server if Visio repeatedly hangs.

In order to cater for the generality of possible failure points for Visio, we recommend that all Publication Servers be configured to periodically detect, and if necessary terminate Visio, if the Visio process is hung.

A hung Visio process will generally be in the "Not Responding" state; however, occasionally a process will be in the Not Responding state temporarily simply because it is genuinely busy (for example if Visio loads a large document it will appear to be Not Responding in Task Manager).

Therefore, for the purposes of terminating processes on the publication server, Triaster recommend the following approach. This will require access to the Server.

Scheduled 'KillVisio' script

The script is intended to work as follows:

If a session of Visio is operating under the publication account, record its Process ID and the amount of CPU Time it has used.
Wait at least 1 minute.
If a session of Visio is operating under the publication account, record its Process ID and the amount of CPU Time it has used.
If the Process IDs and CPU Time from 1) and 3) are the same, then assume Visio is hung and terminate it.

To do this, create a 'KillVisio.cmd' file that contains the following script:

'KillVisio' command script

ECHO OFF
REM ========== THINGS YOU MUST DO BEFORE USING THIS SCRIPT ==============
REM Save the script as a .cmd file
REM Replace the UserAccountName Script Variable with the actual user account that runs the publish
REM Test the script interactively by running in a command window
REM Schedule the script to run once an hour every hour, or, if troubleshooting, once a minute every minute
REM ========== END THINGS YOU MUST DO BEFORE USING THIS SCRIPT ==============

REM ========== SCRIPT COMMENTARY ==============
REM Version 1.1
REM Last Updated May 14th 2012
REM 14/05/2012 Now processes the oldest session of Visio (first listed), as intended.
REM Copyright Triaster Limited 2010. http://www.triaster.co.uk
REM You may use, copy or alter this script provided that at all times you acknowledge Triaster as the original authors.

REM The purpose of this script is to kill the Visio process on the Triaster Publication Server if Visio has not
REM used any processor time on successive executions of this script. By scheduling the script to run automatically
REM it will help prevent the Publication Server locking following a software crash in Visio.
REM How often should it be scheduled to run?
REM We have tested the script on 1 minute intervals and it works well. In a stable environment when Visio crashes rarely
REM we recommend it be scheduled to run every hour.
REM The script works by creating a 2 line text file of the following format:
REM "Image Name","PID","Session Name","Session#","Mem Usage","Status","User Name","CPU Time","Window Title"
REM "VISIO.EXE","2060","Console","0","35,920 K","Running","TriasterPublisher","0:00:08","Microsoft Visio"
REM where TriasterPublisher above is substituted with the user account name of the account that runs the PNPublicationWrapper service.
REM It does this by executing the following command:
REM tasklist /v /fo "CSV" /fi "username eq TriasterPublisher" /fi "imagename eq Visio.exe">%TEMP%\ParameterFile.TXT
REM The first line in the parameter file is the descriptive label of the Visio.exe process parameter, and the 2nd line is the
REM is the value of the parameter.
REM In the case above, the ProcessID is 2060 and the CPU Time is 0:00:08
REM The next time this script is run, the values on the parameter file are read in and stored and a new parameter file is generated.
REM An example of the 2nd output file is shown below:
REM "Image Name","PID","Session Name","Session#","Mem Usage","Status","User Name","CPU Time","Window Title"
REM "VISIO.EXE","2060","Console","0","38,536 K","Running","TRIASTER\Michael Cousins","0:00:19","Microsoft Visio"
REM Note the ProcessID is the same, but the CPU Time has increased to 0:00:19.
REM The 2nd parameter file is then read in and compared with the first. If the ProcessID's match, and the CPU Time has
REM not increased, then the Visio process is presumed to be hung and is killed off.
REM ========== END SCRIPT COMMENTARY ==============

REM ========== SCRIPT VARIABLES ==============
REM To enable the setting of variables within a FOR loop
SETLOCAL ENABLEDELAYEDEXPANSION
REM Change the following to the actual account name and process of interest
SET UserAccountName=TriasterPublisher
SET ProcessName=Visio.exe
REM ========== END SCRIPT VARIABLES ==============

REM ========== MAIN BODY ==============
ECHO Running KillProcess for Process %ProcessName% and user %UserAccountName%
REM Read the parameters from the last time this script was ran, and store in ParameterFileLine
Call :ReadFromFile
ECHO Previous session parameters %ParameterFileLine%
REM Parse the CPU time and ProcessID from ParameterFileLine, and store in CPUTime1 and ProcessID1 respectively.
Call :GetLastValues %ParameterFileLine%
ECHO Previous session parameters ProcessID=%ProcessID1% and CPU Time=%CPUTime1%
REM Write the parameters of the currently running session ....
ECHO tasklist /v /fo "CSV" /fi "username eq %UserAccountName%" /fi "imagename eq %ProcessName%">%TEMP%\ParameterFile.TXT
tasklist /v /fo "CSV" /fi "username eq %UserAccountName%" /fi "imagename eq %ProcessName%">%TEMP%\ParameterFile.TXT
REM .... and read them in again, this time into CPUTime2 and ProcessID2 respectively.
Call :ReadFromFile
Call :GetCurrentValues %ParameterFileLine%
ECHO Current session parameters ProcessID=%ProcessID2% and CPU Time=%CPUTime2%
REM Check the ProcessIDs are the same
IF %ProcessID1%==%ProcessID2% GOTO ProcessIDMatch
Goto :eof

:ProcessIDMatch
REM The ProcessIDs are the same, so, check the CPU Times are the same
ECHO Process IDs match, comparing CPU Times....
IF %CPUTime1%==%CPUTime2% GOTO CPUTimeMatch
Goto :eof
:CPUTimeMatch
REM we have a hung Process - let's kill it.
ECHO CPU Times match, killing process
taskkill /fi "PID eq %ProcessID1%" /f
Goto :eof
REM ========== END MAIN BODY ==============
REM ========== SUBROUTINES ==============
:GetLastValues
REM Read in the ProcessID and CPU Time from the last time this was run.
SET ProcessID1=%2
SET CPUTime1=%8
GOTO :eof

:GetCurrentValues
REM Read in the ProcessID and CPU Time for the current execution.
SET ProcessID2=%2
SET CPUTime2=%8
GOTO :eof

:ReadFromFile
REM Parse the parameter file so that ParameterFileLine2contains the parameter values
SET LineCount=0
FOR /F "tokens=*" %%A IN ('TYPE "%TEMP%\ParameterFile.TXT" 2^>NUL') DO (
SET /A LineCount+=1
SET ParameterFileLine=%%A
IF /I !LineCount! EQU 2 GOTO DoneReading
)

:DoneReading
GOTO :eof
REM ========== END SUBROUTINES ==============
:eof
ENDLOCAL

Ensure that the user account specified in the script is set to the publication user name.

SET UserAccountName=TriasterPublisher

Save the CMD file as, for example:

"C:\Program Files\Triaster\PublicationServer\PublicationWrapper\KillVisio.cmd" (Triaster Server 10.1)

"C:\Triaster\TriasterServer2011\PublicationFiles\KillVisio.cmd" (Triaster Server 11 and later).

Scheduling the script

The command script compares current results with those of the previous run, so should be run at frequent intervals.

Windows Server 2003

Open 'Scheduled Tasks'.

Start > Control Panel > right-click 'Scheduled Tasks' > Open

In 'Scheduled Tasks', double-click 'Add Scheduled Task'.
In the 'Scheduled Task Wizard', click 'Next >'.
Click 'Browse...', select 'KillVisio.cmd', and click 'Open'.
Set the name to 'Kill Visio', 'Perform this task' to 'Daily', and click 'Next >'.
Set 'Start time' to '00:00', 'Perform this task' to 'Every Day', 'Start date' to the current day, and click 'Next >'.
Enter the credentials of the Triaster Services User (aka the Publication User), and click 'Next >'.
Tick 'Open advanced properties...', and click 'Finish'.
On the 'Task' tab of the task dialog, set 'Comments' to 'Checks for and ends dead Visio.exe processes that would interfere with Visio automation'.
On the 'Schedule' tab, click 'Advanced...'.
Set:

Repeat task: TRUE

Every: 1 minute

Duration: 24 hours

and click OK.

On the 'Settings' tab, set 'Stop the task...' to '0 hours 5 minutes'.
Click OK to close the dialog.

Windows Server 2008

Open 'Task Scheduler'.

Start > Administrative Tools > Task Scheduler

In the left-hand pane, expand 'Task Scheduler Library'.
Create a 'Triaster' folder, if necessary.

Click 'Task Scheduler Library'.
In the right-hand pane, click 'New Folder...'.
Name it 'Triaster', and click OK.

In the left-hand pane, click 'Triaster'.
In the right-hand pane, click 'Create Task...'.
In the 'Create Task' dialog, on the 'General' tab, set:

Name: Kill Visio

Description: Checks for and ends dead Visio.exe processes that would interfere with Visio automation.

Run whether user is logged on or not: TRUE

Run with highest privileges: TRUE

On the 'Triggers' tab, click 'New...'.
In the 'New Trigger' dialog, set:

Begin the task: On a schedule

Settings: One time

Start: <Current date> 00:00:00

Repeat task every: <Set to TRUE> 1 minute for a duration of: Indefinitely

Enabled: TRUE

and click OK.

On the 'Actions' tab, click 'New...'.
In the 'New Action' dialog, set:

Action: Start a program

Program/script: <Browse to and select 'KillVisio.cmd'>

and click OK.

On the 'Settings' tab, set:

Stop the task if it runs for longer than: 1 hour

and click OK.

Enter the credentials of the Triaster Services user (aka the Publication User), and click OK.

Additional commands within the Publication Command file

In addition to the scheduled script, insert the following line before any lines that call AnalystEdition.exe in the 'WA Publish.cmd' file found in each site's 'Config Files' directory (Triaster Server 10.1) or create a file called 'KillVisioTasks.cmd' in the 'PublicationFiles' directory (Triaster Server 11.2 and onwards):

taskkill /fi "username eq

TriasterPublisher

" /fi "imagename eq visio.exe" /f

Where you have substituted the

TriasterPublisher

text with the name of your Triaster Server Service Account, which is described here

For example, in a Triaster Server 10.1 'WA Publish.cmd' file:

REM End any Visio.exe process run by the publication user
taskkill /fi "username eq

Steve Jones

" /fi "imagename eq visio.exe" /f

REM Publish the process maps to the site
"c:\program files\triaster\process navigator\analystedition.exe" /TaskSet:"Website
Publication" /p:%ProgressLog% /a:%UserActionsRequiredLog% /e

And in a Triaster Server 11.2 and onwards 'KillVisioTasks.cmd' file:

REM End any Visio.exe process run by the publication user
taskkill /fi "username eq

Steve Jones

" /fi "imagename eq visio.exe" /f

This will terminate any session of Visio that is running (whether or not it is hung).